9 research outputs found
Guaranteed Conformance of Neurosymbolic Models to Natural Constraints
Deep neural networks have emerged as the workhorse for a large section of robotics and control applications, especially as models for dynamical systems. Such data-driven models are in turn used for designing and verifying autonomous systems. This is particularly useful in modeling medical systems where data can be leveraged to individualize treatment. In safety-critical applications, it is important that the data-driven model is conformant to established knowledge from the natural sciences. Such knowledge is often available or can often be distilled into a (possibly black-box) model M. For instance, the unicycle model for an F1 racing car. In this light, we consider the following problem - given a model M and state transition dataset, we wish to best approximate the system model while being bounded distance away from M. We propose a method to guarantee this conformance. Our first step is to distill the dataset into few representative samples called memories, using the idea of a growing neural gas. Next, using these memories we partition the state space into disjoint subsets and compute bounds that should be respected by the neural network, when the input is drawn from a particular subset. This serves as a symbolic wrapper for guaranteed conformance. We argue theoretically that this only leads to bounded increase in approximation error; which can be controlled by increasing the number of memories. We experimentally show that on three case studies (Car Model, Drones, and Artificial Pancreas), our constrained neurosymbolic models conform to specified M models (each encoding various constraints) with order-of-magnitude improvements compared to the augmented Lagrangian and vanilla training methods
Guaranteed Conformance of Neurosymbolic Models to Natural Constraints
Deep neural networks have emerged as the workhorse for a large section of
robotics and control applications, especially as models for dynamical systems.
Such data-driven models are in turn used for designing and verifying autonomous
systems. This is particularly useful in modeling medical systems where data can
be leveraged to individualize treatment. In safety-critical applications, it is
important that the data-driven model is conformant to established knowledge
from the natural sciences. Such knowledge is often available or can often be
distilled into a (possibly black-box) model . For instance, the unicycle
model for an F1 racing car. In this light, we consider the following problem -
given a model and state transition dataset, we wish to best approximate the
system model while being bounded distance away from . We propose a method to
guarantee this conformance. Our first step is to distill the dataset into few
representative samples called memories, using the idea of a growing neural gas.
Next, using these memories we partition the state space into disjoint subsets
and compute bounds that should be respected by the neural network, when the
input is drawn from a particular subset. This serves as a symbolic wrapper for
guaranteed conformance. We argue theoretically that this only leads to bounded
increase in approximation error; which can be controlled by increasing the
number of memories. We experimentally show that on three case studies (Car
Model, Drones, and Artificial Pancreas), our constrained neurosymbolic models
conform to specified models (each encoding various constraints) with
order-of-magnitude improvements compared to the augmented Lagrangian and
vanilla training methods
Memory-Consistent Neural Networks for Imitation Learning
Imitation learning considerably simplifies policy synthesis compared to
alternative approaches by exploiting access to expert demonstrations. For such
imitation policies, errors away from the training samples are particularly
critical. Even rare slip-ups in the policy action outputs can compound quickly
over time, since they lead to unfamiliar future states where the policy is
still more likely to err, eventually causing task failures. We revisit simple
supervised ``behavior cloning'' for conveniently training the policy from
nothing more than pre-recorded demonstrations, but carefully design the model
class to counter the compounding error phenomenon. Our ``memory-consistent
neural network'' (MCNN) outputs are hard-constrained to stay within clearly
specified permissible regions anchored to prototypical ``memory'' training
samples. We provide a guaranteed upper bound for the sub-optimality gap induced
by MCNN policies. Using MCNNs on 9 imitation learning tasks, with MLP,
Transformer, and Diffusion backbones, spanning dexterous robotic manipulation
and driving, proprioceptive inputs and visual inputs, and varying sizes and
types of demonstration data, we find large and consistent gains in performance,
validating that MCNNs are better-suited than vanilla deep neural networks for
imitation learning applications. Website:
https://sites.google.com/view/mcnn-imitationComment: 22 pages (9 main pages
Real-Time Detectors for Digital and Physical Adversarial Inputs to Perception Systems
Deep neural network (DNN) models have proven to be vulnerable to adversarial digital and physical attacks. In this paper, we propose a novel attack- and dataset-agnostic and real-time detector for both types of adversarial inputs to DNN-based perception systems. In particular, the proposed detector relies on the observation that adversarial images are sensitive to certain label-invariant transformations. Specifically, to determine if an image has been adversarially manipulated, the proposed detector checks if the output of the target classifier on a given input image changes significantly after feeding it a transformed version of the image under investigation. Moreover, we show that the proposed detector is computationally-light both at runtime and design-time which makes it suitable for real-time applications that may also involve large-scale image domains. To highlight this, we demonstrate the efficiency of the proposed detector on ImageNet, a task that is computationally challenging for the majority of relevant defenses, and on physically attacked traffic signs that may be encountered in real-time autonomy applications. Finally, we propose the first adversarial dataset, called AdvNet that includes both clean and physical traffic sign images. Our extensive comparative experiments on the MNIST, CIFAR10, ImageNet, and AdvNet datasets show that VisionGuard outperforms existing defenses in terms of scalability and detection performance. We have also evaluated the proposed detector on field test data obtained on a moving vehicle equipped with a perception-based DNN being under attack
Exploring with Sticky Mittens: Reinforcement Learning with Expert Interventions via Option Templates
Long horizon robot learning tasks with sparse rewards pose a significant challenge for current reinforcement learning algorithms. A key feature enabling humans to learn challenging control tasks is that they often receive expert intervention that enables them to understand the high-level structure of the task before mastering low-level control actions. We propose a framework for leveraging expert intervention to solve long-horizon reinforcement learning tasks. We consider option templates, which are specifications encoding a potential option that can be trained using reinforcement learning. We formulate expert intervention as allowing the agent to execute option templates before learning an implementation. This enables them to use an option, before committing costly resources to learning it. We evaluate our approach on three challenging reinforcement learning problems, showing that it outperforms state-of-the-art approaches by two orders of magnitude
Bio-inspired Landing of Quadrotor using Improved State Estimation
This paper presents an improved state estimation technique - a fusion of Monocular SLAM (Simultaneous Localization and Mapping) and INS (Inertial Navigation System). It is utilized in landing a commercially available low cost quadrotor (Parrot AR Drone 2.0) in indoor environments along a trajectory generated by a bio-inspired guidance method. The method is based on Tau theory and facilitates safe and smooth landing of UAVs by closing motion gaps with zero relative velocity and acceleration. A depth camera (Microsoft Kinect) provides a helping hand in very accurate landing towards the end of the quadrotor's trajectory. A dynamic inversion based controller is designed which works as a outer loop controller for the quadrotor. (C) 2018, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved
Improving Neural Network Robustness via Persistency of Excitation
Improving adversarial robustness of neural networks remains a major challenge. Fundamentally, training a neural network via gradient descent is a parameter estimation problem. In adaptive control, maintaining persistency of excitation (PoE) is integral to ensuring convergence of parameter estimates in dynamical systems to their true values. We show that parameter estimation with gradient descent can be modeled as a sampling of an adaptive linear time-varying continuous system. Leveraging this model, and with inspiration from Model-Reference Adaptive Control (MRAC), we prove a sufficient condition to constrain gradient descent updates to reference persistently excited trajectories converging to the true parameters. The sufficient condition is achieved when the learning rate is less than the inverse of the Lipschitz constant of the gradient of loss function. We provide an efficient technique for estimating the corresponding Lipschitz constant in practice using extreme value theory. Our experimental results in both standard and adversarial training illustrate that networks trained with the PoE-motivated learning rate schedule have similar clean accuracy but are significantly more robust to adversarial attacks than models trained using current state-of-the-art heuristics
Towards Alternative Techniques for Improving Adversarial Robustness: Analysis of Adversarial Training at a Spectrum of Perturbations
Adversarial training (AT) and its variants have spearheaded progress in
improving neural network robustness to adversarial perturbations and common
corruptions in the last few years. Algorithm design of AT and its variants are
focused on training models at a specified perturbation strength and
only using the feedback from the performance of that -robust model to
improve the algorithm. In this work, we focus on models, trained on a spectrum
of values. We analyze three perspectives: model performance,
intermediate feature precision and convolution filter sensitivity. In each, we
identify alternative improvements to AT that otherwise wouldn't have been
apparent at a single . Specifically, we find that for a PGD attack at
some strength , there is an AT model at some slightly larger strength
, but no greater, that generalizes best to it. Hence, we propose
overdesigning for robustness where we suggest training models at an
just above . Second, we observe (across various values) that
robustness is highly sensitive to the precision of intermediate features and
particularly those after the first and second layer. Thus, we propose adding a
simple quantization to defenses that improves accuracy on seen and unseen
adaptive attacks. Third, we analyze convolution filters of each layer of models
at increasing and notice that those of the first and second layer
may be solely responsible for amplifying input perturbations. We present our
findings and demonstrate our techniques through experiments with ResNet and
WideResNet models on the CIFAR-10 and CIFAR-10-C datasets